Add Swarming service for Remote Task Gate by jardondiego · Pull Request #5206 · google/clusterfuzz

jardondiego · 2026-03-13T04:50:06Z

Add remote task service for Swarming and include it in Remote Task gate logic.

Summary
This PR integrates Swarming as a first-class remote task execution service within the RemoteTaskGate. It introduces the SwarmingService, which encapsulates the logic for pushing uworker main tasks to Swarming, and updates the
RemoteTaskGate to utilize this service for unified task distribution.

Key Changes

New SwarmingService Implementation: Added a new service that implements the RemoteTaskInterface. It handles task creation by pushing them to Swarming using
swarming.push_swarming_task, providing a standardized interface consistent with Kubernetes and GCP Batch services.
RemoteTaskGate Integration & Refactoring:
- Service Map Expansion: Added SwarmingService to the internal service map of RemoteTaskGate.
- Unified Batch Task Handling: Refactored create_utask_main_jobs to delegate task creation to SwarmingService when the SWARMING_REMOTE_EXECUTION feature flag is enabled. This simplifies the logic by allowing the SwarmingService to
  filter and process swarming-eligible tasks and return the remaining tasks for distribution to other backends.
- Single Task Routing: Updated create_utask_main_job to explicitly route swarming-eligible tasks to the SwarmingService.
RemoteTaskAdapters Update: Added SWARMING to the RemoteTaskAdapters enum. This allows for dynamic control of its execution frequency via feature flags, matching the behavior of other remote execution platforms.

src/clusterfuzz/_internal/swarming/remote_task_service.py

src/clusterfuzz/_internal/swarming/__init__.py

src/clusterfuzz/_internal/swarming/service.py

src/clusterfuzz/_internal/remote_task/remote_task_adapters.py

src/clusterfuzz/_internal/swarming/__init__.py

IvanBM18

LGTM

javanlacerda

Overall looks good, and I think we can merge it once some minor fixes are addressed, and then we can make the other adjustments in the following PRs.

We just need to be sure that the changes will not have any side effect in prod.

src/clusterfuzz/_internal/swarming/service.py

src/clusterfuzz/_internal/remote_task/remote_task_gate.py

javanlacerda

LGTM, I just left a comment on how we could improve the consistency of checking if the tasks is swarming.

src/clusterfuzz/_internal/swarming/service.py

jardondiego · 2026-03-24T23:05:21Z

I merged these changes into dev and deployed them to the development environment where we can see it has not resulted in a regression to the reliability metrics.
https://pantheon.corp.google.com/monitoring/dashboards/builder/9835e451-d912-4a4f-8c67-8165fdde7061;duration=PT12H?project=clusterfuzz-development&e=-13802955&mods=logs_tg_prod&pageState=(%22eventTypes%22:(%22selected%22:%5B%22CLOUD_ALERTING_ALERT%22,%22VM_TERMINATION%22,%22GKE_WORKLOAD_DEPLOYMENT%22%5D))

Also, I monitored the logs and we can only see well known issues in there
https://pantheon.corp.google.com/logs/query;query=severity%3D%22ERROR%22;cursorTimestamp=2026-03-24T23:04:38.276846Z;duration=P1D?referrer=search&e=-13802955&mods=logs_tg_prod&project=clusterfuzz-development

Add Swarming service for Remote Task Gate

da2d5ed

jardondiego force-pushed the swarming-service-remote-task branch 2 times, most recently from 6adf5ef to 241f5b6 Compare March 13, 2026 05:25